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(54) Information search method and system therefor 



(57) A mark management server (103) embeds a 
mark I D and so on in a specified mark image in response 
to a mark request trom a WWW server (102), registers 
information related to a Web page corresponding to this 
mark ID in a mark management DB (2021), and sends 
the mark embedded with the information to the server 
(102). The server (102) attaches this mark to a created 
Web page, and registers the Web page in a Web page 
DB (3021). A client terminal (101) acquires a Web page 
from the server (102), reads information embedded in a , 
mark, and issues a request for a search according to the 
mark to the server (103). The server (103) references 
the DB (2021), and sends information associated with 
a Web page corresponding to a specified mark ID to the 
terminal (101). 



FIG.2 



TO NETWORK 

103 MARK MANAGEMENT 
{ SERVER 



201 PROCESSING 
{ UNIT 



ooo EXTERNAL 
T STORAGE 



-201 1 



PROCESSOR 



"~r~ 

2012 



2013 



20143 



MARK 
CREATING 
PROGRAM 



2014 



20141 

JL 



MARK 
MANAGEMENT 
DS CONTROL 
PROGRAM 



OPERATING 
SYSTEM 



INTERNAL 
STORAGE 
DEVICE 



20142 



-2015 



2021 

\ 





Prnled by Jouve, 75001 PARIS (FR) 



BNSDOCID: <EP_ 



_0938531A2J_> 



1 

Description 

[0001] The present invention relates generally to an 
information search system for searching for multimedia 
data, and more particularly to a system for searching for 
multimedia data based on fixed-pattern data such as a 
mark mounted in the multimedia data. 
[0002] in recent years, WWW (World Wide Web) sys- 
tems have become popular for utilizing networks such 
as the Internet. A WWW system comprises WWW serv- 
ers each for providing a variety of information, and client 
terminals connected to the WWW servers through the 
Internet for receiving information provided therefrom. 
The respective WWW servers open their unique Web 
pages, such that a user can access a Web page by spec- 
ifying URL (Uniform Resource Locater) corresponding 
to the Web page 10 a browser program running on a cli- 
ent terminal. The browser can display multimedia data 
such as text, image data, video data, audio data and so 
on included in the Web page thus acquired on a display 
device of the client terminal. A Web page is described 
using a structural description language called HTML 
(Hyper Text Mark-up Language), so that another Web 
page can be accessed through its URL set in a Web 
page. 

[0003] When a client searches Web pages with a cer- 
tain purpose, the client must access a Web page of a 
WWW server which provides a search service, and re- 
quest for a key word search orspecify a Web page which 
may satisfy the purpose of the client with reference to a 
link list provided by the search service. The link list refers 
to a Web page which collectively has a plurality of listed 
link destinations of Web pages previously selected in 
accordance with an intention of an editor. 
[0004] While a Web page is a copyrighted work, a cli- 
ent, once acquiring the Web page, can readily create 
duplicates and deliver the duplicates through the Inter- 
net. For preventing unauthorized copies of digital data 
sent through the Internet in this way, "electronic water- 
mark" techniques have been proposed for embedding 
I D information, a logo mark or the like of an author within 
a copyrighted work in secrecy. 

[0005] According to the prior art search techniques 
mentioned above, it is not easy to search Web pages 
published by particular authors, Web pages highly eval- 
uated by Web page evaluating organizations, and so on. 
Even if a search service provided on the WWW is uti- 
lized to conduct a key word search, this may result in a 
problematic situation which presents search missing 
that fails to include desired Web pages in a search re- 
sult, and search noise that includes unwanted informa- 
tion. In addition, when existing link lists cannot be uti- 
lized, a client must create his own link list. However, cre- 
ation of a link list involves describing link data to select- 
ed Web pages using the HTML, and so on. This work 
implies a problem in that a large number of work steps 
is required and the created list cannot flexibly support 
changes in Web pages, if any. Furthermore, it is also 



2 

difficult to distinguish normally publisher? Web pages 
from forged or tampered Web pages. 
[0006] Preferably, It is an object of the present inven- 
tion to provide a system for searching only for normal 
5 multimedia data without relying on a key word search or 
creation of a link list, and a program for executing such 
a search. 

[0007] Multimedia data handled by the present inven- 
tion is assumed to be mounted with fixed-pattern data 

to such as a logo mark, which is embedded with code in- 
formation that is difficult for human to perceptually rec- 
ognize. A program on a server for managing such fixed- 
pattern data embeds specified code information in spec- 
ified fixed-pattern data in response to a request from the 

15 outside, and sends the fixed-pattern data to a requester. 
Then, the program registers a database with information 
related to multimedia data mounted with the code infor- 
mation, in correspondence to the code information, 
searches the database based on specified code infor- 

20 mation in response to a search request from a client, 
extracts information related to multimedia data corre- 
sponding to the specified code information, and sends 
the extracted information to a requester. A program on 
the client side in turn decodes code information embed- 

25 ded in fixed-pattern data mounted in multimedia data, 
issues to the management server a search request for 
information related to associated multimedia data based 
on the code information, and displays acquired informa- 
tion related to the associated multimedia data on a dis- 

30 play device. 

BRIEF DESCRIPTION OF THE DRAWINGS 



[0008] 

35 

Fig. 1 is a block diagram illustrating the configura- 
tion of an information search system according to 
an embodiment of the present invention; 
Fig. 2 is a block diagram illustrating the internal con- 
40 figuration of a mark management server 103 in the 
embodiment; 

Fig. 3 is a block diagram illustrating the internal con- 
figuration of a WWW server 1 02 in the embodiment; 
Fig. 4 is a block diagram illustrating the internal con- 
45 figuration of a client terminal 101 in the embodi- 
ment; 

Fig. 5 is a table showing a data structure for a mark 

management DB 2021 in the embodiment; 

Fig. 6 shows an example of information embedded 

50 in a mark; 

Fig. 7 is a flow diagram illustrating a procedure for 
a mark dispense processing in the embodiment; 
Fig. 8 is a flow diagram illustrating a procedure for 
an information search which utilizes the mark in the 

55 embodiment; 

Fig. 9 is a table showing another example of a data 
structure for the mark management DB 2021; 
Fig. 10 shows other examples of information em- 
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bedded in the mark; and 

Fig. 11 is a block diagram illustrating the internal 
configuration of the mark management server 103 
according to another embodiment. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0009] An embodiment of the present invention will 
hereinafter be described with reference to the accom- 
panying drawings. 

[001 0] Fig. 1 is a block diagram illustrating the config- 
uration of a system according to this embodiment. The 
system has a plurality of WWW servers 102 for providing 
information to a network such as the Internet; a plurality 
of client terminals 101 for accessing information provid- 
ed by the WWW servers; and mark management serv- 
ers 103 for managing a variety of marks, wherein they 
are all interconnected through the network. The mark 
herein referred to means still image data, and is as- 
sumed to be image data which allows the user to visually 
understand what the image data means when it is dis- 
played. For example, the mark may be a logo of a credit 
card company, an award mark issued by a Web page 
evaluating organization, a recommendation mark, a 
symbol mark of a self-governing body, or the like. 
[001 1 ] Fig. 2 is a block diagram illustrating the internal 
configuration of the mark management server 103. The 
mark management server 103 comprises a processing 
unit 201 such as PC (personal computer), WS (worksta- 
tion) or the like; an external storage device 202 such as 
HDD (hard disk drive); input devices 203 such as a key- 
board, a mouse and so on; and an output device 204 
such as a display. The processing unit 201 includes an 
interface 2011 for connection to a network; a processor 
201 2 for processing associated with the mark manage- 
ment server 1 03; a memory 201 3 for temporarily storing 
a program; an internal storage device 2014 for storing 
programs and data; and an interface 2015 for connec- 
tion to input and output devices. The external storage 
device 202 stores a mark management DB (database) 
2021 and a mark image DB 2022. The mark manage- 
ment DB 2021 stores information on Web pages to 
which a mark is attached, while the mark image DB 2022 
stores image data associated with respective marks. 
[001 2] The internal storage device 201 4 stores a mark 
creating program 20143, a mark management DB con- 
trol program 20 1 41 and an operating system 201 42. The 
mark creating program 20143 fetches a specified mark 
image from the mark image DB 2022 in response to a 
mark transmission request from the WWW server 102, 
embeds the mark image with information such as an 
identifier of the mark : stores the specified mark identifier 
and information on a Web page to which this mark is 
attached (mounted) in the mark management DB 2021 
through the mark management DB control program 
20141, and transmits the mark image embedded with 
the information to the WWW server 102. 



[0013] The mark management DB control program 
20141 accesses the mark management DB 2021 in ac- 
cordance with a request from the mark creating program 
20143, and accesses the mark management DB 2021 

s in response to a request for associated information re- 
lated to the same mark from a client terminal 1 01 to read 
requested information which is sent to the requesting 
client terminal 101. The processing performed by the 
mark management server 103 in accordance with a 

io mark transmission request from the WWW server 102 
for registering the mark management DB 2021 with in- 
formation on a Web page to which a specified mark is 
attached, and transmitting an mark image embedded 
with information to the WWW server 102 is hereinafter 

75 called the "mark dispense processing". 

[0014] Fig. 3 is a block diagram illustrating the internal 
configuration of the WWW server 102. The WWW serv- 
er 1 02 comprises a processing unit 301 such as PC, WS 
or the like; an external storage device 302 such as HDD; 

20 input devices 303 such as a keyboard, a mouse and so 
on; and an output device 304 such as a display. The 
processing unit 301 includes an interface 3011 for con- 
nection to a network; a processor 3012 for processing 
associated with the WWW server 102; a memory 3013 

25 for temporarily storing a program; an internal storage 
device 3014 for storing programs and data; and an in- 
terface 301 5 for connection with input and output devic- 
es. The external storage device 302 stores a Web page 
DB 3021 , while the Web page DB 3021 stores a plurality 

30 of Web pages managed by the WWW server 102. The 
internal storage device 3014 stores a Web page creat- 
ing program 30143, a Web page DB control program 
301 41 , a mark acquiring program 30144 and an operat- 
ing system 30142. The Web page creating program 

35 30143 creates a Web page in accordance with instruc- 
tions from the user through the input device 303, and 
attaches an acquired mark image to the created Web 
page which is then stored in the Web page DB 3021 
through the Web page DB control program 30141 . The 

40 Web page DB control program 30141 accesses the Web 
page DB 3021 in accordance with a request from the 
Web page creating program 30143, as well as accesses 
the Web page DB 3021 in response to a request from a 
client terminal 1 01 to read a requested Web page there- 

45 from, and sends the read Web page to the client terminal 
101. The mark acquiring program 301 44 issues a mark 
transmission request to the mark management server 
103 in response to a request from the Web page creat- 
ing program 30143, and passes an acquired mark im- 

50 age to the Web page creating program 30143. 

[0015] Fig. 4 is a block diagram illustrating the internal 
configuration of the client terminal 101 . The client termi- 
nal 101 comprises a processing unit 401 such as PC, 
WS or the like; input devices 402 such as a keyboard, 

55 a mouse and so on; and an output device 403 such as 
a display. The processing unit 401 includes an interface 
4011 for connection to a network; a processor 4012 for 
processing associated with the client terminal 101; a 
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memory 401 3 for temporarily storing a program; an in- 
ternal storage device 401 4 for storing programs and da- 
ta; and an interface 401 5 for connection to input and out- 
put devices. The internal storage device 4014 stores a 
browser program 40141 , a search processing program 
40143, and an operating system 40142. The browser 
program 40141 issues a Web page transmission re- 
quest to the WWW server 102, and displays an acquired 
Web page on the output device 403. The search 
processing program 40143 receives a search request 
from the browser program 40141 to extract embedded 
information from a specified mark image, issues a 
search request for associated information related to the 
mark to the mark management server 103, edits ac- 
quired information on associated Web pages, and dis- 
plays the edited information on the output device 403. 
Search request processing in the search processing 
program 40143 issues a search request to the mark 
management server 103 in response to a search re- 
quest for associated information related to the mark 
from the browser program 401 41 . Mark information read 
processing decodes information embedded in a mark 
image. Display/edit processing in turn edits and displays 
a list of acquired Web pages associated with the mark. 
[0016] Fig. 5 is a table showing a data structure for 
the mark management DB 2021 . Each record in the 
mark management DB 2021 is composed of a mark ID 
501 ; a Web page URL 502 and a page name 503. The 
mark ID 501 is an identifier for each mark image stored 
in the mark image DB 2022. The Web page URL 502 is 
the URL of a Web page to which the mark is attached. 
The page name 503 may be a name, entry information 
or the like which shows the contents of the Web page. 
[0017] Fig. 6 shows an example of information em- 
bedded in a mark image. Each mark image is embedded 
with character strings indicative of a mark ID 601 and a 
mark management server address 602 as code infor- 
mation. The mark ID 601 is an identifier of each mark 
image, and the mark management server address 602 
indicates the address of the mark management server 
103 that has embedded information in the mark. 
[0018] Fig. 7 is a flow diagram illustrating a procedure 
for the mark dispense processing. The Web page cre- 
ating program 30143 of the WWW server 102 interacts 
with a manager of a WWW server through the input de- 
vice 303 and the output device 304 to create a Web 
page, and stores the created Web page in the Web page 
DB 3021 through the Web page DB control program 
30141 (step 701). In this event, an access permission 
for the Web page stored in the Web page DB 3021 is 
not granted to ordinary users so that they cannot access 
this Web page. Next, the mark acquiring program 301 44 
issues a request to the mark management server 103 
to send a mark to be attached to the created Web page 
(step 702). This sending request includes a mark ID, a 
Web page URL and a page name. The mark creating 
program 201 43 of the mark management server 1 03 au- 
thenticates the user for this sending request, and then 



acquires a pertinent Web page based on the specified 
URL, and displays the Web page on the output device 
204 for allowing the operator of the mark management 
server 1 03 to confirm the contents of the Web page (step 
5 703). When the operator requests to display a mark im- 
age, the mark creating program 20143 accesses the 
mark image DB 2022 based on a received mark ID, and 
displays the mark image associated with the mark ID on 
the output device 204. After the operator determines 
10 that the mark may be dispensed to the Web page, for 
which the mark has been requested, without problem, 
when a mark dispensing instruction is issued through 
the input device 203, the mark creating program 20143 
embeds the mark ID 601 and the mark management 
is server address 602 in the requested mark image (step 
704). A technique for embedding particular information 
in a mark image is known as "digital watermark" which 
is described, for example, in Nikkei Electronics, pp. 
100-107, No. 683, 1997. Next, the mark creating pro- 
20 gram 20143 registers the mark ID 501 of the dispensed 
mark image, and the URL 502 and the page name 503 
of the requested Web page in the mark management 
DB 2021 (step 705). Next, the mark creating program 
20143 transmits the mark image embedded with the in- 
25 formation to the WWW server 102 (step 706). 

[0019] The mark acquiring program 30144 of the 
WWW server 102, which has received the mark image, 
passes it to the Web page creating program 30143. The 
Web page creating program 30143 attaches this mark 
30 image to a created Web page to create a marked Web 
page (step 707). For attaching the mark, the Web page 
and the mark image are displayed on the output device 
304, and the mark is attached at a location specified by 
the manager on the Web page. Next, the pertinent Web 
35 page previously registered on the Web page DB 3021 
is updated with the marked Web page (step 708). Then, 
the access permission for the updated Web page is 
changed to be accessible by ordinary users. 
[0020] Fig. B is a flow diagram illustrating a procedure 
40 for information search utilizing a mark. It should be noted 
that the information search herein mentioned refers to 
the processing for searching for Web pages which have 
attached thereon the same mark as a mark attached on 
a certain Web page, dispensed from the mark manage- 
rs ment server 103. The browser program 40141 of the cli- 
ent terminal 1 01 requests a pertinent WWW server 1 02 
to transmit a Web page based on a URL specified 
through the input device 402 (step 801). The Web page 
DB control program 30141 of the WWW server 102 re- 
50 ceives this transmission request, searches the Web 
page DB 3021 (step 802), retrieves the specified Web 
page, and transmits the contents of that Web page to 
the client terminal 101 (step 803). Assume that the re- 
trieved Web page has a mark attached thereon, dis- 
ss pensed from the mark management server 103. The 
browser program 40141 of the client terminal 101 dis- 
plays the received Web page on the output device 403 
(step 804). The user can visually recognize what the 
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mark displayed on the output device 403 represents, but 
cannot visually recognize information embedded in the 
mark. When a search request is made for associated 
information by specifying a mark attached on a Web 
page through the input device 402, the search request 
processing is started in the search processing program 
40143 (step 805). This processing in the search 
processing program 40143 starts the mark information 
read processing. 

[0021] The mark information read processing reads 
information (the mark ID 601 and the mark management 
server address 602) embedded in a mark image (step 
806). The technique for reading text information from a 
mark image is known as a part of the "digital watermark" 
technique. Next, the search request processing re- 
quests the mark management server 103 to search for 
associated information based on the information em- 
bedded in the mark (step 807). Upon receipt of the re- 
quest, the mark management DB control program 
20141 of the mark management server 103 searches 
the mark management DB 2021 to acquire the Web 
page URLs 502 and the page names 503 of Web pages 
(having the same mark ID 501 as the received mark) to 
which the same mark as the specified one has been dis- 
pensed (step 808). Next, the mark management DB 
control program 20141 transmits a list of the Web page 
URLs 502 and the page names 503, which have been 
acquired as the associated information, to the client ter- 
minal 101 (step 809). The search processing program 
401 43 of the client terminal 101 edits the received infor- 
mation with its display/edit processing, and displays the 
edited information on the output device 403 through the 
browser program 40141 (step 810). 
[0022] When any Web page within the associated in- 
formation is requested for display through the input de- 
vice 402 (step 811), the search processing program 
401 43 passes the URL of a selected Web page to the 
browser program 40141, and the browser program 
40141 requests the WWW server 102 associated with 
the URL to transmit the Web page (step 812). The Web 
DB control program 30141 of the WWW server 102 
searches the Web page DB 3021 to retrieve the speci- 
fied Web page, and sends the contents thereof to the 
client terminal 101 (step 813). The browser program 
40141 of the client terminal 101 displays the received 
Web page on the output device 403 (step 814). 
[0023] As described above, since the mark manage- 
ment sever 1 03 holds mark dispense information on the 
mark management DB 2021 implemented therein, the 
mark management server 1 03 can specify a destination 
to which the same mark has been dispensed, and there- 
fore allows the mark ID 501 to be used as a key for 
searching for the associated information. 
[0024] While the user could duplicate a mark image 
within a Web page acquired from the WWW server 102 
and attach the duplicate to a Web page created thereby 
without authorization, such a Web page is not registered 
in the mark management server 1 03 and therefore is not 



regarded as an object of a search for the associated in- 
formation as described above. In addition, for a Web 
page on which a mark thus forged is attached, an ille- 
gally duplicated mark can be detected by referring to the 

s mark management server 1 03 based on the mark I D and 
the Web page URL thereof. Furthermore, since the in- 
formation embedded in a mark image is difficult to 
tamper, it is difficult for an unauthorized user to modify 
information embedded in a normal mark image to infor- 

10 mation convenient to him. It is therefore possible to pre- 
vent the action of distributing false marks forged from 
an authoritative mark, which has been famous, over a 
network. 

[0025] As described above, since the client terminal 

is 101 decodes information embedded in a mark, this in- 
formation is likely to be acquired and abused by the user. 
As a method of ensuring the security for embedded in- 
formation, a method pi encrypting the embedded infor- 
mation is contemplated. For example, the mark man- 

20 agement server 103 may encrypt information to be em- 
bedded in a mark using a secret key, prior to the em- 
bedding of the information in the mark, in the processing 
for embedding information in a mark (step 704). The cli- 
ent terminal 101 requests the mark management server 

25 103 to send a public key, and the mark management 
server 103 responsively sends the public key to the cli- 
ent terminal 101. The client terminal 101 acquires the 
encrypted code information embedded in the mark from 
the mark management server 103, and decrypts the 

30 code information embedded in the mark using the public 
key before utilizing the code information. While the us- 
ers can relatively easily acquire this public key, the users 
can be managed through user authentication. 
[0026] When the same mark has been dispensed to 

35 a large number of Web servers, attribute information 
may be added to each record in the mark management 
DB 2021 such that the attribute information is utilized for 
searching for a mark. For example, an entry called "in- 
tended age" may be provided as attribute information, 

40 wherein a search result may be narrowed up by speci- 
fying the attribute as a search condition if Web pages 
are classified into those "for 20 years or more" and those 
"for under 20 years". Also, as illustrated in Fig. 9, an en- 
try called "order" 504 may be added as attribute infor- 

45 mation in each record of the mark management DB 
2021 in addition to the mark ID 501 , the Web page URL 
502 and the page name 503. In this case, the mark man- 
agement DB control program 201411 may refer to the 
order 504 in the mark management DB 2021 to sort per- 

50 tinent records according to the order 504 used as a key, 
and then send a search result to the client terminal 1 01 , 
thereby making it possible to display the search result 
in the order set in the mark management DB 2021 . 
[0027] Of course, the mark management DB control 

55 program 20141 may transfer pertinent records in the 
mark management DB 2021 as they are to the client ter- 
minal 101, so that the client terminal 101 can sort the 
records according to the order 504 used as a key. With 
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such a method, Web pages having the same mark are 
browsed by the user of the client terminal 101 in an order 
intended by a mark dispenser. 

[0028] Alternatively, a mark may be embedded with 
the position in the order at which an associated Web 
page is browsed, instead ot providing the order 504 in 
each record in the mark management DB 2021 . Fig. 10 
shows exemplary records which have information la- 
belled front 603 and behind 604 embedded therein in 
addition to the mark ID 601 and the mark management 
server address 602, as information to be embedded in 
a mark. Each line in Fig. 10 corresponds to each of Web 
pages having the same mark, wherein front 603 sets the 
URL and page name of a Web page positioned imme- 
diately in front of a current Web page, while behind 604 
sets the URL and page name of a Web page positioned 
immediately behind the current Web page. With this 
method, the mark management server 103 is only re- 
quired to send the client terminal 101 the Web page URL 
502 and the page name 503 of the record at the head 
of a group of Web pages having the same mark, and the 
client terminal 101 can directly access a Web page po- 
sitioned in front of or behind a displayed Web page. 
[0029] While the foregoing embodiment has shown 
an example of searching for associated information for 
a single mark, the client terminal 101 may simultane- 
ously issue search requests to a plurality of marks, so 
that a plurality of marks may be dispensed respectively 
from different mark management servers 1 03 without in- 
curring any problem. 

[0030] Also, while in the foregoing embodiment, the 
mark management server 1 03 notifies the client terminal 
101 of destinations to which a specified mark has been 
dispensed, the mark management server 103 may also 
transfer the contents of the mark management DB 2021 
to the client terminal 1 01 utilizing techniques such as file 
transfer or the like, in response to desire to know which 
marks can be otherwise seen, or to make a search for 
associated information for another mark. Fig. 11 illus- 
trates the internal configuration of a client terminal 101 
which handles a transferred mark management DB 
2021 . An internal storage device 401 4 is provided there- 
in with a mark information storage area 401 45 for storing 
the contents of the mark management DB 2021 , and a 
search processing program 40143 is provided therein 
with mark information search processing as a program 
for searching the mark information storage area 40145 
[0031] While in the foregoing embodiment, the mark 
management DB 2021 is located on the mark manage- 
ment server 103, it may be located on any other data- 
base server which may be accessible by the mark man- 
agement server 103, instead of locating it on the mark 
management server 103. Also, while in the foregoing 
embodiment, the mark management server 103 per- 
forms the mark dispense processing and the search 
processing, the mark management server 103 may be 
dedicated only to the mark dispense processing, while 
the search processing may be assigned to another serv- 



er having duplicated contents of the mark management 
DB 2021 , in view of the loading on the servers. In this 
case, the address of a search server may be embedded 
in a mark instead of embedding the address of the mark 
5 management server 103. Also, in this case, a plurality 
of search servers may be provided corresponding to the 
single mark management server 103 for distributing the 
load of the search processing. 

[0032] While the foregoing embodiment has been de- 

io scribed in connection with marks implemented by still 
image data, the present invention is not limited to this 
particular type of marks. When perceptually recogniza- 
ble data formed of a single or a combination of a plurality 
of character data (text information), still image data, vid- 

f£ eo data (moving image data) and audio data is called 
multimedia data : the present invention can be applied 
to fixed-pattern data that may constitute a portion of 
such multimedia data as a mark. For example, the 
present invention can be applied to the case where the 

20 multimedia data is media data such as a movie including 
video data and audio data, and a mark is a fixed-pattern 
data that constitutes a portion of the video data or the 
audio data. As an example of embedding secret infor- 
mation in character data, there is such a method that 

25 adds a predetermined number of blanks between char- 
acters which constitutes text information, or behind the 
character string. Also, a technique is known for embed- 
ding secret information in audio information so as not to 
be audibly recognizable. When video data or audio data 

30 is applied as a mark, this may exceed the concept of a 
normal mark, or the concept of attaching a mark to a 
Web page. In this case, however, what is equivalent to 
a Web page may be corresponded to the body of multi- 
media data including entry information such as a title, a 

35 publisher or the like; the mark to fixed-pattern data 
mounted in the multimedia data body; and the mark ID 
to an identifier of this fixed-pattern data (an identifier of 
a copyright owner or the like). 

[0033] According to the present invention as de- 
40 scribed above, since information on associated multi- 
media data stored in a database is extracted on the ba- 
sis of code information embedded in fixed-pattern data 
mounted in the multimedia data, targeted normal multi- 
media data can be searched for without causing search 
45 missing or search noise associated with a key word 
search or without requiring a large number of steps 
which would be required to create a link list. 



50 Claims 

1, An information search processing method for reg- 
istering information related to multimedia data hav- 
ing code information associated therewith in a da- 
tabase (2021), said method comprising the steps of; 

(a) in response to a request from the outside, 
embedding perceptually unrecognizable code 
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information in perceptually recognizable fixed- 
pattern data mounted in multimedia data in- 
cluding at least one of character data, still im- 
age data, moving image data and audio data, 
and sending the multimedia data embedded 5 
with the code information to a requester (102); 

(b) registering information related to said mul- 
timedia data mounted with said code informa- 
tion in the database, in correspondence to said 
code information: and 10 

(c) in response to a search request from theout- 
side, searching said database based on spec- 
ified code information to extract information on 
said multimedia data corresponding to the 
specified code information, and sending the ex- 15 
tracted information to a requester. 

2. An information search processing method accord- 
ing to claim 1, wherein said tixed-pattern data, em- 
bedded with said code inlormation, is a mark image 20 
in the form of a still image. 

3. An information search method for requesting to 
search for associated multimedia data based on 
perceptually unrecognizable code information em- 2s 
bedded in perceptually recognizable fixed-pattern 
data mounted in multimedia data including at least 
one of character data, still image data : moving im- 
age data and audio data, said method comprising 

the steps of: 30 

(a) decoding said code information embedded 
in said fixed-pattern data related to the multi- 
media data; 

(b) issuing a search request for information re- 35 
lated to the associated multimedia data to an 
external processing unit (103) based on said 
code information, said external processing unit 
having information related to the associated 

mu Itimedia data corresponding to said code in- 40 
formation as a database (2021); and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 

45 

4. An information search processing method for re- 
questing to search for associated multimedia data 
based on visually unrecognizable code information 
embedded in a visually recognizable mark image 
mounted in multimedia data including at least one so 
of character data, stilt image data, moving image 
data and audio data, said method comprising the 
steps of: 



associated multimedia data based on said code 
information embedded when said mark image 
was specified; and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 

5. An information search system having a first server 
(102) for providing multimedia data including at 
least one of character data, still image data, moving 
image data and audio data, a client terminal (101) 
for receiving the multimedia data provided by said 
first server, and a second server (103) for embed- 
ding perceptually unrecognizable code information 
in perceptually recognizable fixed-pattern data 
mounted in the multimedia data and for managing 
information related to the multimedia data mounted 
with said code information, wherein: 

said first server includes means (30141) re- 
sponsive to a request from said client terminal 
for sending specified multimedia data to said 
client terminal, means (30144) for requesting 
said second server to send said fixed-pattern 
data embedded with said code information, and 
means (30143) for mounting said fixed-pattern 
data in corresponding multimedia data; 
said second server includes means (201 43) re- 
sponsive to a request from said first server for 
mounting specified code information in speci- 
fied fixed-pattern data and sending the fixed- 
pattern data to said first server, means (20143) 
for registering information related to multimedia 
data mounted with said code information in a 
database, and means (20141) responsive to a 
search request from said client terminal for 
searching said database based on specified 
code information to extract information related 
to multimedia data corresponding to the speci- 
fied code information and sending the extracted 
information to said client terminal; and 
said client terminal includes means (401 41 ) for 
requesting said first server to acquire the mul- 
timedia data mounted with the fixed-pattern da- 
ta, means (40143) for decoding code informa- 
tion embedded in acquired fixed-pattern data, 
means (401 43) for issuing to said second serv- 
er a search request for information related to 
multimedia data associated with the code infor- 
mation based on the decoded code informa- 
tion, and means (40143) for displaying ac- 
quired information related to the associated 
multimedia data on a display device (403). 



(a) displaying said mark image on a display de- 55 
vice (403); 

(b) issuing to an external processing unit (103) 
a search request for information related to the 



A computer program embodied on a computer- 
readable medium to perform method steps for reg- 
istering in a database (2021) information related to 
multimedia data having code information associat- 
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ed therewith, said method steps comprising the 
steps of: 

(a) in response to a request from the outside, 
embedding perceptually unrecognizable code s 
information in perceptually recognizable fixed- 
pattern data mounted in multimedia data in- 
cluding at least one of character data, still im- 
age data, moving image data and audio data, 
and sending the multimedia data embedded 10 
with the code information to a requester (102); 

(b) registering information related to said mul- 
timedia data mounted with said code informa- 
tion in the database, in correspondence to said 
code information; and is 

(c) in response to a search request from the out- 
side, searching said database based on spec- 
ified code information to extract information on 
said multimedia data corresponding to the 
specified code information, and sending the ex- 20 
tracted information to a requester. 

7. A computer program embodied on a computer- 
readable medium to perform method steps for re- 
questing to search for associated multimedia data 25 
based on perceptually unrecognizable code infor- 
mation embedded in perceptually recognizable 
fixed-pattern data mounted in multimedia data in- 
cluding at least one of character data, still image 
data, moving image data and audio data, said meth- 30 
od steps comprising the steps of: 

(a) decoding said code information embedded 
in said fixed-pattern data related to the multi- 
media data; 35 

(b) issuing a search request for information re- 
lated to the associated multimedia data to an 
external processing unit (103) based on said 
code information, said external processing unit 
having information related to the associated 40 
multimedia data corresponding to said code in- 
formation as a database (2021); and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 45 

8. A computer program embodied on a computer- 
readable medium to perform method steps for re- 
questing to search for associated multimedia data 
based on visually unrecognizable code information so 
embedded in a visually recognizable mark image 
mounted in multimedia data including at least one 

of character data, still image data, moving image 
data and audio data, said method steps comprising 
the steps of: ss 

(a) displaying said mark image on a display de- 
vice (403); 



(b) issuing to an external processing unit (103) 
a search request for information related to the 
associated multimedia data based on said code 
information embedded when said mark image 
was specified; and 

(c) displaying acquired information related to 
the associated multimedia data on adisplay de- 
vice (403). 
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